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ABSTRACT: 

This paper is a formal analysis of whether generalized phrase struc- 
ture grammar's (GPSG) weak context-free generative power will allow it to 
achieve three of its central goals: (1) to characterize all and only the natural 
language grammars, (2) to algorithmically determine membership and gen- 
erative power consequences of GPSGs, and (3) to embody the universalism 
of natural language entirely in the formal system. I prove that "= S*?" is 
undecidable for GPSGs and, on the basis of this result and the unnaturalness 
of S*, I argue that GPSG's three goals and its weak context-free genera- 
tive power conflict with each other: there is no algorithmic way of knowing 
whether any given GPSG generates a natural language or an unnatural one. 
The paper concludes with a diagnosis of the result and suggests that the 
problem might be met by abandoning the weak context-free framework and 
assuming substantive constraints. 



This report describes research done in part at the Artificial Intelligence Laboratory 
of the Massachusetts Institute of Technology. Support for the Laboratory's artificial 
intelligence research has been provided in part by the Advanced Research Projects 
Agency of the Department of Defense under Office of Naval Research contract 
N00014-80-C-0505. This paper will be presented at the 1986 ACL Conference in 
June. 

©Eric Sven Ristad, 1986 



1 OVERVIEW 1 

1 Overview 

Three central goals of work in the generalized phrase structure grammar 
(GPSG) linguistic framework, as stated in the leading book "Generalized 
Phrase Structure Grammar" Gazdar et al (1985) (hereafter GKPS), are: 
(1) to characterize all and only the natural language grammars, (2) to al- 
gorithmically determine membership and generative power consequences of 
GPSGs, and (3) to embody the universalism of natural language entirely in 
the formal system, rather than by statements made in it. 1 

These pages formally consider whether GPSG's weak context-free gener- 
ative power (wcfgp) will allow it to achieve the three goals. The centerpiece 
of this paper is a proof that it is undecidable whether an arbitrary GPSG 
generates the nonnatural language S*. On the basis of this result, I ar- 
gue that GPSG fails to define the natural language grammars, and that 
the generative power consequences of the GPSG framework cannot be al- 
gorithmically determined, contrary to goals one and two. 2 In the process, 
I examine the linguistic universalism of the GPSG formal system and ar- 
gue that GPSGs can describe an infinite class of nonnatural context-free 
languages. The paper concludes with a brief diagnosis of the result and sug- 
gests that the problem might be met by abandoning the weak context-free 
generative power framework and assuming substantive constraints. 



'GKPS clearly outline their goals. One, "to arrive at a constrained metalanguage 
capable of denning the grammars of natural languages, but not the grammar of, say, the 
set of prime numbers."(p.4). Two, to construct an explicit linguistic theory whose formal 
consequences are clearly and easily determinable. These 'formal consequences' include 
both the generative power consequences demanded by the first goal and membership 
determination: GPSG regards languages "as collections whose membership is definitely 
and precisely specifiable."(p.l) Three, to define a linguistic theory where "the universalism 
[of natural language] is, ultimately, intended to be entirely embodied in the formal system, 
not expressed by statements made in rt."(p.4, my emphasis) 

2 The proof technique make use of invalid computations, and the actual GPSG con- 
structed is so simple, so similar to the GPSGs proposed for actual natural languages, 
and so flexible in its exact formulation that the method of proof suggests there may be no 
simple reformulations of GPSG that avoid this problem. The proof also suggests that it is 
impossible in principle to algorithmically determine whether linguistic theories based on 
a wcfgp framework (e.g. GPSG) actually define the natural language grammars. 
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1.1 The Structure of GPS G Theory 

A generalized phrase structure grammar contains five language-particular 
components (immediate dominance (ID) rules, metarules, linear precedence 
(LP) statements, feature co-occurrence restrictions (FCRs), and feature 
specification defaults (FSDs)) and four universal components: a theory of 
syntactic features, principles of universal feature instantiation, principles of 
semantic interpretation, and formal relationships among various components 
of the grammar. 3 

The set of ID rules obtained by taking the finite closure of the metarules 
on the ID rules is mapped into local phrase structure trees, subject to prin- 
ciples of universal feature instantiation, FSDs, FCRs, and LP statements. 
Finally, these local trees are assembled to form phrase structure trees, which 
are terminated by lexical elements. 

The essence of GPSG is the constrained mapping of ID rules into local 
trees. The constraints of GPSG theory subdivide into absolute constraints 
on local trees (due to FCRs and LP-statements) and relative constraints on 
the rule to local tree mapping (stemming from FSDs and universal feature 
instantiation). The absolute constraints are all language-particular, and 
consequently not inherent in the formal GPSG framework. Similarly, the 
relative constraints, of which only universal instantiation is not explicitly 
language-particular, do not apply to fully specified ID rules and consequently 
are not strongly inherent in the GPSG framework either. 4 In summary, 
GPSG local trees are only as constrained as ID rules are: that is, not at all. 

The only constraint strongly inherent in GPSG theory (when compared 
to context-free grammars (CFGs)) is finite feature closure, which limits the 
number of GPSG nonterminal symbols to be finite and bounded. 5 



3 This work is based on current GPSG theory as presented in GKPS. The reader is 
urged to consult that work for a formal presentation and thorough exposition of current 
GPSG theory. 

4 I use "strongly inherent" to mean "unavoidable by virtue of the formal framework." 
Note that the use of problematic feature specifications in universal feature instantiation 
means that this constraint is dependent on other, parochial, components (e.g. FCRs). 
Appropriate choice of FCRs or ID rules will abrogate universal feature instantiation, thus 
rendering it implicitly language particular too. 

5 This formal constraint is extremely weak, however, since the theory of syntactic fea- 
tures licenses more than 10 774 syntactic categories. See Ristad(l986) for a discussion. 
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1.2 A Nonnatural GPSG 

Consider the exceedingly simple GPSG for the nonnatural language E*, 
consisting solely of the two ID rules 

S^{},H\e 

This GPSG generates local trees with all possible subcategorization spec- 
ifications — the SUBCAT feature may assume any value in the non-head 
daughter of the first ID rule, and S generates the nonnatural language S*. 

This exhibit is inconclusive, however. We have only shown that GKPS 
— and not GPSG — have failed to achieve the first goal of GPSG theory. 
The exhibition leaves open the possibility of trivially reformalizing GPSG 
or imposing ad-hoc constraints on the theory such that I will no longer be 
able to personally construct a GPSG for S*. 



2 Undecidability and Generative Power in GPSG 

That "= S*?" is undecidable for arbitrary context-free grammars is a well- 
known result in the formal language literature (see Hopcraft and Ullman(1979:201- 
203)). The standard proof is to construct a pushdown automata (PDA) that 
accepts all invalid computations of a Turing machine (TM) M. From this 
PDA an equivalent CFG G is directly constructible. Thus, L(G) = S* if 
and only if all computations of M are invalid, i.e. L(M) = 0. The latter 
problem is undecidable, so the former must be also. 

No such reduction is possible for a proof that "= S*?" is undecidable 
for arbitrary GPSGs. In the above reduction, the number of nonterminals 
in G is a function of the size of the simulated TM M. GPSGs, however, 
have a bounded number of nonterminal symbols, and as discussed above, 
that is the essential difference between CFGs and GPSGs. 

Only weak generative power is of interest for the following proof, and the 
formal GPSG constraints on weak generative power are trivially abrogated. 
For example, exhaustive constant partial ordering (ECPO) — which is a 
constraint on strong generative capacity — can be done away with for all 
intents and purposes by nonterminal renaming, and constraints arising from 
principles of universal feature instantiation don't apply to fully instantiated 
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ID rules. 

First, a proof that "= E*?" is undecidable for context-free grammars 
with a very small number of terminal and nonterminal symbols is sketched. 
Following the proof for CFGs, the equivalent proof for GPSGs is outlined. 

2.1 Outline of a Proof for Small CFGs 

Let L( xy \ be the class of context-free grammars with at least x nonterminal 
and y terminal symbols. I now sketch a proof that it is undecidable of 
an arbitrary CFG G € -k^.y) whether L(G) = S* for some x,y greater 
than fixed lower bounds. The actual construction details are of no obvious 
mathematical or pedagogical interest, and will not be included. The idea 
is to directly construct a CFG to generate the invalid computations of the 
Universal Turing Machine (UTM). This grammar will be small if the UTM is 
small. The "smallest UTM" of Minsky(1967:276-281) has seven states and 
a four symbol tape alphabet, for a state-symbol product of 28 (!). Hence, 
it is not surprising that the "smallest Gutm" that generates the invalid 
computations of the UTM has seventeen nonterminals and two terminals. 

Observe that if a string w is an invalid computation of the universal Tur- 
ing machine M = (Q, S, T, 6,qo, B, F) on input x, then one of the following 
conditions must hold. 

1. w has a "syntactic error," that is, w is not of the form xi#a;2# • • • #£m#> 
where each a;,- is an instantaneous description (ID) of M. Therefore, 
some x,- is not an ID of M . 

2. xi is not initial; that is, xi £ qoE* 

3. x m is not final; that is x m £ T* fT* 

4. Xi >-*m (xi+i) R is false for some odd t 

5. (xi) R >-*m £,+1 is false for some even t 

Straightforward construction of Gutm will result in a CFG containing on 
the order of twenty or thirty nonterminals and at least fifteen terminals (one 
for each UTM state and tape symbol, one for the blank-tape symbol, and one 
for the instantaneous description separator "#"). Then the subgrammars 
which ensure that (x,) B i-i-a* x,+i is false for some even * and that x t - i-t- M 
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(xi + i) R is false for some odd t may be cleverly combined so that nonterminals 
encode more information, and so on. 

The final trick, due to Albert Meyer, reduces the terminals to 2 at the 
cost of a lone nonterminal by encoding the n terminals as log n = k-bit words 
over the new terminal alphabet {0, 1}, and adding some rules to ensure that 
the final grammar could generate S* and not (E 4 )*. The productions 

N 4 -> OL4IL4 I OOL4 I OIL4 I HI4 I • • • 

are added to the converted CFG G' UTM , which generates a language of 
the form 

L 4 ->• 0000 I 0001 I 0010 I ... I e I L4L4 

Where L\ generates all symbols of length 4, and N4 generates all strings 
not of length mod k, where k = 4 (i.e. all strings of length 1,2,3 mod 4). 
Deeper consideration of the actual Gutm reveals that the N4 nonterminal 
is also eliminable. 

Note that all the preceding efforts to reduce the number of nonterminals 
and terminals increase the number of context-free productions. This symbol- 
production tradeoff becomes clearer when one actually constructs Gutm- 

Suppose the distinguished start symbol for Gutm is Sutm- Then we 
form a new CFG consisting of all productions of the form 

s - {Q - go}{S p - (M)}{N4 u L 4 } 
and the one production 

S — *■ Sutm 

where (M) is the length p encoding of an arbitrary TM M , and L4, N4 
are as defined above. 

This ensures that strings whose prefix is a qo{M) n will be generated start- 
ing from 5 if and only if they are generated starting from Sutm'- that is, 
they are invalid computations of the UTM on M . 
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2.2 Some Details for L (l iV) and GPSG 

Let the nonterminal symbols T,Q, and S in the following CFG portion 
generate the obvious terminal symbols corresponding to the equivalent UTM 
sets. B is the terminal blank symbol. 

Then, the following sketched CF productions generate the IDs of M such 
that X{ H-»jvf (xi+i) R is false for some odd i. 

The S4 and S5 nonterminals are used to locate the even and odd i IDs 
X{ of w. S k generates the language {r U #}*. 

5 4 — ► TS4 I #55 I #S oc idSok 

55 —* T55 I #54 I #S even S k 

Sodd —> "Sl# 

Si -* TSiT \S2\Se\S7 

Sq —* TSq I TS3 

S7 — ► ^r 1 iSsr 

S 2 -» HaES 3 TbT 

where o 7^ b, both in S 
S 2 -» aqbS 3 {T s - pea} if S(q, b) = (p, c, R) 
aqbS s {T 3 - cap} if S(q, b) = (p, c, L) 

5 2 -» aqB#B{r 3 - pea} if S(q, B) = (p, c, R) 

aqB#B{T s - cap} if S(q, B) = {p, c, L) 

5 3 -> TS S T I QB#BTT | SB#JBr 

£1 and 5*2 must generate a false transition for odd t, while 53 need 
not generate a false transition and is used to pad out the IDs of w. The 
nonterminals 56, SV accept IDs with improperly different tape lengths. The 
first 52 production accepts transitions where the tape contents differ in a 
bad place, the second S2 production accepts invalid transitions other than 
at the end of the tape, and the third 52 accepts invalid end of the tape 
transitions. Note that the last two 52 productions are actually classes of 
productions, one for each string in T 3 — pea, T 3 — cap, .... 

The GPSG for "= £*?" is constructed in a virtually identical fashion. 
Recall that the GPSG formal framework does not bar us from construct- 
ing a grammar equivalent to the CFG just presented. The ID rules used 
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in the construction will be fully specified so as to defeat universal feature 
instantiation, and the construction will use nonterminal renaming to avoid 
ECPO. 

Let the GPSG category C be fully specified for all features (the actual 
values don't matter) with the exception of, say, the binary features GER, 
NEG , NULL and POSS. Arrange those four features in some canonical order, 
and let binary strings of length four represent the values assigned to those 
features in a given category. For example, C[0100] represents the category C 
with the additional specifications ([-GER] , [+NEG] . [-NULL], [-P0SS]). 
We replace S odd by C[0000], Si by C[0001], S 2 by C[0010], S 3 by C[0011], 
S e by C[0100], and S 7 by C[0101]. The nonterminal T is replaced by three 
symbols of the form C[llxx], one for each linear precedence to which T 
conforms. Similarly, S is replaced by two symbols of the form C[100x]. The 
ID rules, in the same order as the CF productions above (with a portion of 
the necessary LP statements) are: 



C[0000] -» C[0001]# 

C[0001] -► C[1100]C[0001]C[1101] I C[0010] I C[0100] I C[010l] 

C[0100] -» C[1100]C[0100] I C[1100]C[0011] 

C[0101] -»• C[0101]C[1101] I C[0011]C[1101] 

C[0010] -» C[lOOO]aC[100l]C[001l]C[llOl]6C[lllO] 

where a 7^ b, both in S 
C[0010] -»■ ag6C[0011]{r 3 - pea} if S(q, b) = (p, c, R) 

ag6C[001l]{r 3 - cap} if %, b) = (p, c, L) 
C[0010] -» aqB#B{T 3 - pea} if 6(q, B) = (p, c, R) 

aqB#B{T 3 - cap} if S(q, B) = (p, c, L) 

C[0011] -♦ C[1100]C[0011]C[1101] I 
QB#BC[1100]C[1101] I 
C[1000]£#£C[1100] 

C[1100] < C[0001],C[0011],C[0100],C[0101] < C[1101] 
C[1000] < a < C[1001] < C[0011] < C[1110] 

While the sketched ID rules are not valid GPSG rules, just as the 
sketched context-free productions were not the valid components of a context- 
free grammar, a valid GPSG can be constructed in a straightforward and 
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obvious manner from the sketched ID rules. There would be no metarules, 
FCRs or FSDs in the actual grammar. 

The last comment to be made is that in the actual Gutm> on ty the 
number of productions is a function of the size of the UTM. The UTM is 
used only as a convincing crutch, because only a small, fixed number of 
nonterminals are needed to construct a CFG for the invalid computations of 
any arbitrary Turing Machine. 

3 Interpreting the Result 

The preceding pages have shown that the extremely simple nonnatural lan- 
guage E* is generated by a GPSG, as is the more complex language Lie 
consisting of the invalid computations of an arbitrary Turing machine on an 
arbitrary input. Because Lie is a GPSG language, "= 2*?" is undecidable 
for GPSGs: there is no algorithmic way of knowing whether any given GPSG 
generates a natural language or an unnatural one. So, for example, no al- 
gorithm can tell us whether the English GPSG of GKPS really generates 
English or S*. 

The result suggests that goals 1, 2, 3 and the context-free framework 
conflict with each other. Weak context-free generative power allows both 
S* and Lie, yet by goal 1 we must exclude nonnatural languages. Goal 2 
demands it be possible to algorithmically determine whether a given GPSG 
generates a desired language or not, yet this cannot be done in the context- 
free framework. Lastly, goal 3 requires that all nonnatural languages be 
excluded on the basis of the formal system alone, but this looks to be im- 
possible given the other two goals, the adopted framework, and the technical 
vagueness of "natural language grammar." 

The problem can be met in part by abandoning the context-free frame- 
work. Other authors have argued that natural language is not context-free, 
and here we argue that the GPSG theory of GKPS can characterize context- 
free languages that are too simple or trivial to be natural, e.g. any finite 
or regular language. 6 The context-free framework is both too weak and too 



6 While 'natural language grammar' is not defined precisely, recent work has demon- 
strated empirically that natural language is not context-free, and therefore GPSG theory 
will not be able to characterize all the human language grammars. See, for example, 
Higginbotham(1984), Shieber(1985), and Culy(1985). For counterarguments, see Pul- 
lum(1985). Nash(1980), chapter 5, discusses the impossibility of accounting for free word 
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strong — it includes nonnatural languages and excludes natural ones. More- 
over, CFL's have the wrong formal properties entirely: natural language is 
surely not closed under union, concatenation, Kleene closure, substitution, 
or intersection with regular sets! 7 In short, the context-free framework is the 
wrong idea completely, and this is to be expected: why should the arbitrary 
generative power classifications of mathematics (formal language theory) be 
at all relevant to biology (human language)? 

Goal 2, that the naturalness of grammars postulated by linguistic the- 
ory be decidable, and to a lesser extent goal 3, are of dubious merit. In 
my view, substantive constraints arising from psychology, biology or even 
physics may be freely invoked, with a corresponding change in the meaning 
of "natural language grammar" from "mentally-representable grammar" to 
something like "easily learnable and speakable mentally-representable gram- 
mar." There is no a priori reason or empirical evidence to suggest that 
the class of mentally representable grammars is not fantastically complex, 
maybe not even decidable. 8 

One promising restriction in this regard, which if properly formulated 
would alleviate GPSG's actual and formal inability to characterize only the 
natural language grammars, is strong nativism — the restrictive theory that 
the class of natural languages is finite. This restriction is well motivated 
both by the issues raised here and by other empirical considerations. 9 The 
restriction, which may be substantive or purely formal, is a formal attack on 
the heart of the result: the theory of undecidability is concerned with the 
existence or nonexistence of algorithms for solving problems with an infinity 



order languages (e.g. Warlpiri) using ID/LP grammars. I focus on the goal of character- 
izing only the natural language grammars in this paper. 

7 The finite, bounded number of nonterminals allowed in GPSG theory plays a linguistic 
role in this regard, because the direct consequence of finite feature closure is that GPSG 
languages are not truly closed under union, concatenation, or substitution. 

"See Chomsky(1980:120) for a discussion. 

9 Note that invoking finiteness here is technically different from hiding intractability 
with finiteness. Finiteness is the correct generalization here, because we are interested in 
whether GPSG generates nonnatural languages or not, and not in the computational cost 
of determining the generative capacity of an arbitrary GPSG. A finiteness restriction for 
the purposes of computational complexity is invalid because it prevents us from properly 
using the tools of complexity theory to study the computational complexity of a problem. 
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of instances. Furthermore, the restriction may be empirically plausible. 10,11 

The author does not have a clear idea how GPSG might be restricted 
in this manner, and merely suggests strong nativism as a well-motivated 
direction for future GPSG research. 
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supporting this research. 
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language, I argue that GPSG' s three goals and its weak context-free 
generative power conflict with each other: there is no algorithmic way 
of knowing whether any given GPSG generates a natural language or an 
unnatural one. The paper concludes with a diagnosis of the result and 
suggests that the problem might be met by abandoning the weak 
context-free framework and assuming substantive constraints. 
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